Summary

• TCP latency proxy (latency_proxy.go) that injects configurable bidirectional delay between DuckDB/DuckLake and the metadata PostgreSQL, enabling analysis of how network latency (simulating cross-region RDS) affects transaction conflict rates
• DUCKGRES_BENCH_LATENCIES env var — comma-separated latency sweep, e.g. 0ms,25ms,50ms,100ms. Default: 0ms (backward-compatible, no behavior change)
• Per-latency duckgres harnesses — each latency level gets a fresh duckgres server with a TCP proxy in front of the metadata PostgreSQL
• JSON report schema — adds metadata_latency_ms per metric and latencies_tested_ms to the report for downstream comparison
• justfile recipes: bench-ducklake-latency (single version, latency sweep) and bench-ducklake-full-matrix (version × latency 2D matrix)
• Version matrix script passes through DUCKGRES_BENCH_LATENCIES for 2D analysis

Usage

# Quick latency sweep on current version
just bench-ducklake-latency 0ms,25ms,50ms,100ms

# Full version × latency matrix
DUCKGRES_BENCH_LATENCIES=0ms,50ms,100ms just bench-ducklake-matrix

# Or directly
DUCKGRES_BENCH_LATENCIES=0ms,50ms go test -v -run TestDuckLakeConcurrentTransactions -timeout 600s ./tests/integration/...

How it works

The latency proxy sits between DuckDB's DuckLake extension and the metadata PostgreSQL:

DuckDB → DuckLake ext → [TCP Latency Proxy (+Xms per direction)] → Metadata PostgreSQL

Each read/write to the proxy gets a time.Sleep(latency) before forwarding, so a 50ms setting adds ~100ms RTT overhead — matching cross-region RDS latency.

Architecture

• LatencyProxy is a plain Go TCP proxy (no external dependencies)
• For 0ms latency, the default test harness is used directly (zero overhead)
• For non-zero latencies, a fresh TestHarness is created per latency level with the proxy wired in
• All harnesses share the same metadata PostgreSQL and MinIO containers
• Test structure: TestDuckLakeConcurrentTransactions/latency_0ms/concurrent_inserts_same_table

Test plan

• TestLatencyProxy — zero latency forwarding, measurable latency (~100ms RTT for 50ms one-way), close terminates connections
• All existing integration tests pass
• go vet ./... clean
• Compilation succeeds
• CI passes
• DuckLake concurrency tests with DUCKGRES_BENCH_LATENCIES=0ms,50ms (requires DuckLake infra)

🤖 Generated with Claude Code